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Abstract— This Research-to-Practice paper, a work-in- 
progress, describes work supported by the First in the World 
program at three different universities in California. A 
consortium of three California State Universities (CSUs)— San 
José State University, California State University- Los Angeles, 
and Cal Poly Pomona -- have a four-year grant from the U.S 
Department of Education First-in-the-World (FITW) program. 
Surveys of students revealed that a major challenge to success is 
course bottlenecks — impasses where they cannot enroll in a 
course they need to make progress toward their degrees or when 
they cannot successfully complete a course and move forward. 
All three campuses have large numbers of high-need and under- 
represented students (URM) and URM students are over- 
represented among students who receive low grades in bottleneck 
courses. To address course bottlenecks, the flipped classroom 
approach has been implemented in seven gateway STEM courses 
collaboratively across the partner campuses. This paper targets 
faculty and administrators interested in promoting and 
implementing the flipped classroom pedagogy at their 
institutions. It provides a brief overview of the target courses and 
the impact of the curricular changes thus far. In addition, a 
description of the in-depth Calculus study of the flipped 
classroom approach across the three campuses is discussed. 


Keywords— underrepresented students, active learning, flipped 
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I. INTRODUCTION 


This paper reports on a collaborative project of three 
California State University (CSU) campuses—San José State 
University (SJSU), California State University- Los Angeles 
(CSULA), and Cal Poly Pomona (CPP)—to improve the 
retention rates and success of students in gateway STEM 
courses. The CSU is the largest and most diverse university 
systems in the country. This project builds upon the 
collaborative environment in the CSU by engaging faculty 
from three different campuses in a rigorous collaboration and 
evaluation of pedagogy in STEM education. This project 
incorporates a flipped classroom approach in high-failure 
undergraduate gateway STEM courses at these three CSUs 
and evaluates if the teaching style is effective at decreasing 
failure rates of students. 

San José State University, located in San José, California, 
is the oldest campus in the CSU system. The university offers 
bachelors and masters degrees in 134 areas of study and two 
doctorates to more than 32,000 undergraduate and graduate 
students in seven colleges. The Davidson College of 
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Engineering and College of Science prepare students for 
graduate training and a global and Silicon Valley workforce. 

CSULA in Los Angeles, California was founded in 1947 
and focuses on providing excellent education to its eight 
colleges. CSULA has one of the most diverse student 
populations of any college or university in the nation. Building 
on this diversity, CSULA graduates constitute a major 
leadership force in Los Angeles. 

CPP, located in Pomona, California is one of only seven 
polytechnic universities in the nation; its College of 
Engineering graduates 1 of every 14 engineers in the state. 
Enrolling 24,000 students, the university’s focus on “learn by 
doing” is evident in curricular designs that blend theory and 
practice. Students are afforded multiple opportunities to 
collaborate with faculty on research projects. 

There are many commonalities among our three 
institutions. All three are designated Hispanic-serving 
institutions (HSI) and Asian American Native American 
Pacific Islander serving institutions (AANAPISI). At each 
institution, many students are first generation, under- 
represented minorities (URM) and/or low income. 64% of 
SJSU students, 80.8% of CSULA students, and 66% of CPP 
students qualify for some kind of financial aid (loans, grants, 
scholarship, etc.) and 38% of SJSU students, 75% of CSULA 
students, and 44% of CPP students receive Pell grants. Figure 
1 summarizes the undergraduate student ethnicity as of Fall 
2014 for STEM majors in our three universities. 

This project targets undergraduate gateway STEM 
courses with high failure rates. Recent reports highlight the 
need for curricular reforms that promote problem solving, 
collaboration, and “deep learning” [1 ]. Active learning 
approaches have proven to be especially crucial for learning 
experiences of high-needs students [2]. High-need students are 
those at risk of educational failure or otherwise in need of 
special assistance and support, such as adult learners, working 
students, part-time students, students from low-income 
backgrounds, students of color, first-generation students, 
students with disabilities, and students who are English 
learners [3]. The vast majority of students at SJSU, CSULA, 
and CPP fall into one or more of these categories. This project 
utilized the definition of active learning proposed by Freeman 
et al. [4], “Active learning engages students in the process of 
learning through activities and/or discussion in class, as 
opposed to passively listening to an expert.” Active learning 
increases academic self-confidence, motivation to persist in 
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academic pathways, motivation to complete one’s academic 
study, and likelihood of pursuing advanced study [5 ]. 
Significant research shows student learning can be improved 
when instructors move from a teacher-centered approach to a 
student-centered, interactive approach [6]. 


SJSU CSULA CPP 

HC |%Total| HC |%Total| HC |%Total 
AmlInd 4, 0.1%) 16 4% 17) 0.2% 
Black 111} 2.6% 149) 3.4%] 275) 2.9% 
Asian | 1,715} 40.3%} 828} 18.6%| 2,428} 25.9% 
Pac Isl 33} 0.1% 7.1%) 21 0.2% 
Hispanic] 837} 19.6% 2,526] 56.9%] 3,261} 34.8% 
White | 901} 21.1%] 374) 8.4%| 2,136) 22.8% 
Foreign} 309] 7.3%} 297, 6.7%| 444) 4.7% 
Other | 350) 8.2%] 246 5.5%| 802) 8.5% 

Total | 4,260 4,443 9,149 

Figure 1. Demographic information for SJSU, 
CSULA, and CPP shows that all three universities 
have large minority populations 


This project focuses on one active learning technique, the 
flipped classroom. In a flipped classroom, the standard 
teacher-centered lecture is replaced with a student-focused 
approach that involves strategies implemented outside the 
classroom [7]. Students use online resources to learn the 
material before class, allowing them to learn at their own pace 
and be better prepared for in-class work [8]. Classroom time is 
reserved for active learning including problem-based learning 
and practice activities [9]. Although the flipped classroom has 
been implemented in different ways, the idea is to move a 
student’s acquisition of basic knowledge out of the classroom 
and use class time for higher-level activities that build upon 
basic knowledge [10]. This approach is shown to have a 
positive effect on student learning outcomes, especially in 
regards to STEM courses, which are rated as some of the most 
challenging and alienating university classes [11]. 

This project implements the flipped classroom across 
STEM courses at three large CSUs. The project builds upon 
extensive cross-field research, proven models for increasing 
student engagement and participation in STEM courses, and 
assesses the effectiveness of these strategies. In addition to the 
curricular development, the project investigates the use of 
Faculty Learning Communities to support faculty in curricular 
development and build a culture of change to expand the use 
of active learning strategies in STEM classes. 


II. FACULTY LEARNING COMMUNITIES 


Wright and Sunal [12] found that a key factor in 
educational innovation is for faculty to be willing participants 
in innovations. Despite being subject experts, few university 
faculty have formal training in pedagogy [13]. Several recent 
reports [14] highlight the need for curricular reforms to 
provide undergraduate education that engages both students 
and instructors in activities that promote problem solving, 
collaboration, and “deep learning” [15]. 


Guiding principles for faculty development are based 
upon the social-cognition model and supported by the science 
of learning perspective as described in several National 
Research Council publications, including How People Learn: 
Brain, Mind, Experience, and School [16]. The curricular 
revision in this project linked faculty across the three CSU 
campuses in order to generate new thinking about models of 
instruction [17]. In Year 1, core faculty responsible for 
converting the targeted courses and project leadership from all 
three campuses attended an intensive three-day workshop. 
During the workshop, core faculty met with the PI and the 
campus leads to clarify the project goals, timelines, and 
responsibilities for their course redesign. 

Each campus has a Faculty Learning Community (FLC) 
to facilitate collaboration on this project. In addition, cross- 
campus disciplinary FLCs were developed so faculty in each 
discipline could collaborate. FLCs provide an arena in which 
colleagues have the time and opportunities to reflect on their 
teaching, their discipline, their institution, and themselves. 
FLCs inspire faculty to think beyond their discipline and into 
the realm of intellectual and interpersonal connections [18]. 
These FLCs are aligned with the project’s social-cognition 
theory of change; FLCs foster constructive interactions and 
allow faculty to explore their mental models about teaching. 
According to Senge [19], an individual’s mental models or 
cognitive frameworks shape and frame behavior and “limit us 
to familiar ways of thinking and acting.” The FLCs were 
designed to foster faculty leadership and empower faculty to 
be change agents in their departments, at their institutions, and 
among STEM faculty across the CSU [20]. Each campus has a 
FLC coordinator to facilitate FLC activities. Faculty on each 
campus meet face-to-face and digitally to discuss progress in 
their curricular development, explore new ideas, and increase 
knowledge of active learning. The design of this project 
facilitates cross-institutional collaboration and development of 
both discipline-focused and campus-centered FLCs. 

Coordinated FLC activities have included collaborative 
sessions where faculty learn about new developments in 
flipped learning, reflect on their current flipping challenges, 
provide feedback to colleagues on flipped materials, and 
engage in development of tools that support flipping efforts. 
Participants in these activities may be all grant faculty across 
the campuses, grant faculty on only one campus, or discipline 
faculty across the campuses. Activities that require 
coordination across the campuses are handled through a 
variety of online tools. Project faculty communicate in their 
discipline-focused FLCs through phone calls, email, video 
conferences, message boards, and chat rooms. The types of 
communication tools used for each disciplinary FLC have 
varied; what are consistent across all disciplinary FLCs are 
regular communication and a sense of community among the 
members. FLC activities and sessions remain appropriately 
fluid in response to faculty needs. 


III. FLIPPED CLASSROOM CURRICULUM DEVELOPMENT 


Courses for this project were chosen based on high levels 
of students who received grades lower than a “C” as of Fall 
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2014. Research at SJSU has [21] showed that students who 
received a grade below a C in a freshmen or sophomore 
STEM class rarely passed the subsequent class in the 
sequence. In addition, it is well established that many 
engineering students, for example, are “lost” before 
engineering faculty interact with them because they become 
discouraged in preparatory courses [22]. This project targeted 
freshmen and sophomore gateway STEM courses. 

Curricular development for this project was phased in 
over three years. In Year | (2016-2017), faculty from the three 
CSUs implemented the flipped classroom approach in two 
classes: Calculus I and General Physics I. In Year 2 (2017- 
2018), Computer Science I and Introduction to Circuits were 
flipped. In Year 3 (2018-2019), the flipped classroom will be 
implemented in three final courses: General Physics II, 
Discrete Math, and Statics. 

Calculus I and General (Calculus-based) Physics I were 
chosen as the first courses to be flipped because of the 
literature from CSU and other institutions on flipping these 
courses. General Physics I was identified by the CSU as one 
of the CSU’s 22 high demand-low success courses [23]. In the 
CSU, 1/5 of the students in the 22 high demand-low success 
courses received grades lower than a C. In response, the CSU 
created a Physics eAcademy focused on teaching Physics in a 
flipped classroom model [24]. This initiative served as a 
foundation for development of General Physics I in FITW. 

In 1998, Richard Hake collected data on 2,084 students 
from 14 introductory physics courses taught by traditional 
instruction methods and compared these results to classes 
using active learning for 4,458 students in 48 courses [25]. His 
research showed an increase in learning nearly two standard 
deviations higher when using interactive engagement 
methods. Hake’s research is supported by research by 
ENGAGE that shows that faculty-student interactions increase 
student retention rates in STEM courses [26]. 

Calculus I was chosen as a first-year course for the grant 
because of promising literature that suggests flipping Calculus 
can lead to increased student learning gains. According to the 
College Board [27], Calculus is an impediment to student 
success in STEM careers. In Fall 2010, 325,000 students 
enrolled nationwide in a university Calculus I course; of this 
number, 27% received a D or F or withdrew. The numbers of 
students unsuccessful in Calculus I at some project campuses 
is higher than the nationwide numbers. Since Calculus I is 
often the first mathematics class for STEM students, this 
course was deemed very important to serve as a foundation for 
student retention and success in subsequent courses. 


IV. RESULTS OF THE FLIPPED CLASSROOM PROJECT 


Preliminary results presented here are not evaluated 
based on any demographic, student preparedness, or student 
performance outside of the given class. Such detailed 
analyses are reserved for the in-depth Calculus study and 
presentation/publication by the faculty teaching the course. 
Results from the first term of Year 1 courses (Calculus I and 
Physics I) were presented at the 2017 ASEE Annual 
Conference and Exposition [28]. 


In an additional term of Calculus I and Physics I at 
CSULA, results continued to show positive impact of using 
the flipped model. Calculus I was taught in 16 sections (size 
range: 17-26), with one section being in the flipped model 
(size: 23). A comparison of grades showed that 65.2% of 
students in the flipped section received a passing grade of 
“C” or better, as compared to only 62.9% of students in the 
traditional section. The GPA of the class in the flipped 
section was 2.23, compared to the other sections with GPAs 
ranging from 1.33-3.0. Students in the flipped class did 
achieve a grade of “C-” or better at a far greater percentage 
than students in the traditional classes (78.3% versus 
64.4%). Physics I at CSULA had similar positive results 
across four Physics I sections (Size Range: 26-72), with one 
section using the flipped model (Size: 49). In the flipped 
section, 93.9% of students received a grade of “C” or better 
as compared to 76% in the traditional sections. The GPA for 
the flipped section was 2.93, the highest among all sections 
(GPA Range: 1.98-2.93; 1.98-2.54 for traditional sections). 

Initial preliminary results for Year 2 courses reflect the 
first semester of instruction in Fall 2017 using the flipped 
classroom approach. Introduction to Circuits courses across 
the three campuses have historically had among the highest 
failure rates. While content in Circuits is similar across 
campuses, variation in other structural changes in the 
courses and class size make cross-campus comparisons 
difficult. Introduction to Programming has even greater 
variation across the campuses, including programming 
language used, departmental ownership of the courses, class 
size, and term offerings. In addition to the change in 
pedagogy to the flipped classroom, CSULA transitioned 
from a quarter system to a semester system during the 
evaluation semester, which could have an influence on 
flipping a course for both the instructors and students. Data 
collection will continue in Spring 2018 and after to enable 
greater inference on the effectiveness of the flipped 
classroom in all courses across all project campuses. 

Results show that flipping the classroom generally has a 
positive impact on passing rates when comparing similar 
classes. All sections of Circuits at SJSU had a significant 
number of structural and curricular changes in addition to 
utilizing the flipped classroom and other active learning 
pedagogy. Results show that those structural changes, 
combined with the curricular changes, saw nearly 10% more 
students receiving A’s or B’s when compared to sections of 
similar size in previous semesters taught by the same 
instructors. At CSULA, two flipped sections were offered in 
Fall 2017, though they varied considerably in size (Sizes: 35 
and 83), making evaluation challenging. In order to evaluate 
the effect of the flipped classroom, three sections of similar 
size (Size: 28-35) taught by the same instructor in a 
previous semester were compared to the smaller flipped 
section in Fall 2017. When comparing the Circuits sections 
of similar size, the flipped section shows positive results, 
with a higher percentage of students passing the course 
(62.9% versus 60.5%) and having higher achievement at all 
grade levels (A-B-C). Circuits at CPP was taught in four 
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sections (Size: 29-32), with one using flipped classroom 
pedagogy. The flipped classroom resulted in 11.3% more 
students passing the course with a C or better and no 
students receiving an F or withdrawing from the course. 

Introduction to Programming varies in content, the 
department in which the course is housed, and the grading 
structure on the three campuses; this contributes to 
challenges in evaluating the effectiveness of the flipped 
approach. Introduction to Programming (CS 46A) at SJSU 
is housed in the Computer Science Department. The course 
was taught in the flipped classroom pedagogy only in 
Summer 2017, with a second offering planned for Summer 
2018. Results between the Summer 2017 offering and an 
academic year offering by a different instructor show that 
the flipped classroom section was less successful than the 
academic year section despite being smaller (Size: 25 
compared to 80). Only 64% of students received a C or 
better in the summer flipped section, compared to 70% in 
the traditional academic year section. Grant personnel are 
awating additional data from a second summer session, as 
well as evaluating potential variables that may have 
contributed to the lower results, including faculty 
evaluations and student preparedness for the course. 

Introduction to Programming (EE 2450) is housed in 
the Electrical Engineering department at CSULA. CSULA 
has a large number of students, approximately 20-30%, who 
take the course as Credit/No Credit, which does not enable a 
strong comparison to the other campuses nor does it enable 
a full evaulation of student success because lower grades are 
not reported. The course was taught with three sections 
using traditional instruction (Size: 22-26) and one section 
using the flipped classroom (Size: 30). Initial results for the 
flipped section show reduced success in the course (70% 
versus 77.5%). A large number of students received a grade 
of “No Credit” (NC) in the flipped section (N = 9), which 
was relatively high when compared to the students who 
received “NC” other sections (N = 3, 5, and 6). 

Introduction to Programming at CPP is taught in two 
different departments: the Computer Science Department 
(CS 141) and the Electrical and Computer Engineering 
department (ECE 114). In Fall 2017, three sections of CS 
141 were offered, two using traditional pedagogy (Size: 34- 
36) and one with the flipped classroom (Size: 37). Students 
were significantly more successful in the flipped section, 
with 94.6% of students passing the course with a C or better 
compared to 84.1% in the traditional sections. Students also 
had higher achievement in the flipped section, with over 
51% receiving a grade of “A/A-” compared to only 29% in 
the traditional sections. In Fall 2017, five sections of ECE 
114 were offered, with one section taught with the flipped 
classroom. The traditional class sections ranged in size from 
14 to 27, while the flipped classroom was 26. While 
students passed the course in similar percentages in both the 
traditional and flipped sections (95.7% versus 96.2%), 
nearly 12% more students earned an A or a B in the flipped 
section. 


V. CALCULUS IMPACT STUDY 


The external evaluator for this project is conducting a 
summative evaluation of the flipped classroom model. 
Calculus I is the focus of the summative evaluation because it 
will have benefitted from two years of development. The 
evaluation will specifically address: Does being in a class 
using the flipped model improve student outcomes in 
calculus? What implementation factors moderate the 
effectiveness of the flipped approach? Here “effectiveness” is 
measured by student performance outcomes in calculus, both 
in terms of grades and a study-specific pre- and post-test. 

The design is a randomized controlled trial. Participating 
instructors agree that their course sections will be randomly 
assigned, each semester, to use instruction as they normally 
would teach (control condition) or use a flipped approach 
(treatment condition). Instructors teaching a control section 
can do whatever they normally would, but not to use “flipped” 
techniques. For treatment sections, instructors use flipping in 
the class. Instructors can elect to participate by teaching one or 
two sections. Instructors who agree to teach two sections of 
the course agree that they will teach one section as control 
and one as treatment. They track their activities in a journal so 
that their work can be reviewed by the evaluator who can then 
consider any overlapping that may have occurred. 

Ideally, instructors participate for 2 academic terms 
(Spring 2018 and Fall 2018 for SJSU and CSULA). Since 
CPP is converting from a quarter structure to a semester 
structure in Fall 2018, the Calculus study for CPP will be Fall 
2018 and Spring 2019. Campus FLC coordinators will work 
with a local campus Calculus | faculty leader to offer eight 
hours of “Do the Flip!” training for calculus instructors. The 
time was spread across several meetings in Fall 2017 for those 
who began to flip classes in Spring 2018. There will be 
another workshop in summer 2018 for those calculus 
instructors who will be new to flipping in Fall 2018. 


VI. CONCLUSIONS 


The three universities have worked collaboratively to 
improve instruction methods used in seven gateway STEM 
courses identified to have high failure rates across the 
campuses. The flipped classroom model is being phased in 
over three years on the campuses through collaborative 
efforts by faculty to develop materials to support 
knowledge acquisition outside of class time and engage 
students in higher-order applications such as problem 
solving and peer instruction during class time. Results show 
increased pass rates for students in most flipped classroom 
sections when compared to sections taught with traditional 
methods. The final draft of this paper will include 
preliminary results on the multi-campus Calculus study. 
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